Providing augmented reality based on third party information

ABSTRACT

A profile set by a particular user from a social network application may be used to determine a view provided by the mobile electronic device to a user of the mobile electronic device. The particular user in a captured video of the current view is detected and augmented in real time to mimic a virtual character in an augmented video.

CROSS REFERENCE TO RELATED APPLICATION

This application is a continuation of U.S. patent application Ser. No.13/052,359, filed Mar. 21, 2011, that is herein incorporated byreference as if fully set forth.

FIELD OF INVENTION

This application is related to an apparatus and method for providingenhanced augmented reality and an advanced display device. An augmentedor mixed reality world may be provided to a user based on other user orthird party information, such as from a profile. A display device isprovided with adaptive transparent and non-transparent portions foraugmented reality and any other application.

BACKGROUND

Augmented or mixed reality is a way of superimposing or overlayingcomputer generated information onto a physical or real-worldenvironment. The computer generated information may be visual, auditory,or tactile. Thus far, the computer generated information has been basedon first person requests or configurations.

Display devices are commonplace in electronic devices such as mobiledevices, cellular phones, personal digital assistants, smartphones,tablet personal computers (PCs), laptop computers, televisions,monitors, touchscreens, digital picture frame, or the like. Currently,display devices may be based on liquid crystal, plasma, light emitting,organic light emitting, or field effect technologies using ridged orflexible substrates. The next generation display devices may betransparent or clear opening new possibilities and applications, such asfor augmented reality. However because of the transparency, viewinginformation in daylight or other environments is less optimal thanopaque or darken background display devices.

Therefore, it is desirable to have augmented reality based on another'spoint of view and a transparent display device with controllabletransparency.

SUMMARY

An apparatus and method for providing augmented or mixed realityenvironments based on other user or third party information, such asfrom a profile, is disclosed. Also, an apparatus and method forproviding a transparent display device with adaptive transparency isdisclosed. The transparent display device with adaptive transparency maybe used to generate the augmented reality environment.

BRIEF DESCRIPTION OF THE DRAWINGS

A more detailed understanding may be had from the following description,given by way of example in conjunction with the accompanying drawingswherein:

FIG. 1 is a diagram of an electronic device;

FIGS. 2a-2c are diagrams of configurations for providing a displaydevice with adaptive transparency;

FIG. 3 is a diagram of a displayed augmented reality environment;

FIG. 4 is a process for providing a display device with adaptivetransparency; and

FIG. 5 is a process for providing an augmented reality environment.

DETAILED DESCRIPTION

The present invention will be described with reference to the drawingfigures wherein like numerals represent like elements throughout. Forthe methods and processes described below the steps recited may beperformed out of sequence in any order and sub-steps not explicitlydescribed or shown may be performed. In addition, “coupled” or“operatively coupled” may mean that objects are linked between zero ormore intermediate objects. Also, any combination of the disclosedfeatures/elements may be used in one or more embodiments. When usingreferring to “A or B”, it may include A, B, or A and B, which may beextended similarly to longer lists.

U.S. Patent Publication No. 2010-238114 and U.S. patent application Ser.No. 12/706,205 are herein incorporated by reference as if fully setforth and may be used in combination with the given examples to providea display device with portions that are selectively elevated, indented,or texturized. Also in the examples forthcoming, display devices mayadaptively emit ultrasound, ultrasonic, acoustic, or radio waves toprovide elevated, indented, or texturized sensation to an object near adisplay device.

Elevation or elevated sensations describe different sensations that maybe caused to an object or person with a controlled flexible displaydevice layer or using ultrasound at a predetermined or random distancefrom a display or electronic device. As an example, the relativedistance of the object may be one or more millimeters to several meters,as desired. Indenting may be a configuration where an object or personis given a sensation around its perimeter while giving little sensationto the inner area of the object or person. Texturizing or texturingdescribes a process where an electronic device using a controlledflexible display device layer or ultrasound over air to provide,simulate, or mimic friction, pulsing sensation, pulsating sensation,variable smoothness, variable thickness, coarseness, fineness,irregularity, a movement sensation, bumpiness, or rigidness that issensed by or detectable by an object or person.

FIG. 1 is a diagram of an object device 100 that may be a wirelesssubscriber unit, user equipment (UE), mobile station, smartphone, pager,mobile computer, cellular telephone, telephone, personal digitalassistant (PDA), computing device, surface computer, tablet computer,monitor, general display, versatile device, digital picture frame,appliance, automobile computer system, vehicle computer system, part ofa windshield computer system, television device, home appliance, homecomputer system, laptop, netbook, tablet computer, personal computer(PC), wireless sensor, an Internet pad, digital music player,peripheral, add-on, an attachment, virtual reality glasses, mediaplayer, video game device, head-mounted display (HMD), helmet mounteddisplay (HMD), glasses, goggles, or any electronic device for mobile orfixed applications.

Object device 100 comprises computer bus 140 that couples one or moreprocessors 102, one or more interface controllers 104, memory 106 havingsoftware 108, storage device 110, power source 112, and/or one or moredisplays controller 120. In addition, object device 100 comprises anelevation, indenting, or texturizing controller 121 to providesensations to an object or person located near one or more displaydevices 122.

One or more display devices 122 can be configured as a plasma, liquidcrystal display (LCD), light emitting diode (LED), field emissiondisplay (FED), surface-conduction electron-emitter display (SED),organic light emitting diode (OLED), or flexible OLED display device.The one or more display devices 122 may be configured, manufactured,produced, or assembled based on the descriptions provided in U.S. PatentPublication Nos. 2007-247422, 2007-139391, 2007-085838, or 2006-096392or U.S. Pat. No. 7,050,835 or WO Publication No. 2007-012899 all hereinincorporated by reference as if fully set forth. In the case of aflexible or bendable display device, the one or more electronic displaydevices 122 may be configured and assembled using organic light emittingdiodes (OLED), liquid crystal displays using flexible substratetechnology, flexible transistors, field emission displays (FED) usingflexible substrate technology, or the like.

One or more display devices 122 can be configured as a touch ormulti-touch screen display using resistive, capacitive, surface-acousticwave (SAW) capacitive, infrared, strain gauge, optical imaging,dispersive signal technology, acoustic pulse recognition, frustratedtotal internal reflection, or magneto-strictive technology, asunderstood by one of ordinary skill in the art. One or more displaydevices 122 can also be configured as a three dimensional (3D),electronic paper (e-paper), or electronic ink (e-ink) display device.

Coupled to one or more display devices 122 may be pressure sensors 123.Coupled to computer bus 140 are one or more input/output (I/O)controller 116, I/O devices 118, global navigation satellite system(GNSS) device 114, one or more network adapters 128, and/or one or moreantennas 130. Examples of I/O devices include a speaker, microphone,keyboard, keypad, touchpad, display, touchscreen, wireless gesturedevice, a digital camera, a digital video recorder, a vibration device,universal serial bus (USB) connection, a USB device, or the like. Anexample of GNSS is the Global Positioning System (GPS).

Object device 100 may have one or more motion, proximity, light,optical, chemical, environmental, moisture, acoustic, heat, temperature,radio frequency identification (RFID), biometric, face recognition,image, photo, or voice recognition sensors 126 and touch detectors 124for detecting any touch inputs, including multi-touch inputs, for one ormore display devices 122. Sensors 126 may also be an accelerometer, anelectronic compass (e-compass), gyroscope, a 3D gyroscope, or the like.One or more interface controllers 104 may communicate with touchdetectors 124 and I/O controller 116 for determining user inputs toobject device 100. Coupled to one or more display devices 122 may bepressure sensors 123 for detecting presses on one or more displaydevices 122.

Ultrasound source/detector 125 may be configured in combination withtouch detectors 124, elevation, indenting, or texturizing controller121, one or more display devices 122, pressure sensors 123, or sensors126 to project or generate ultrasound waves, rays, or beams to an objectto simulate elevated, indented, or texturized sensations, recognizeinputs, or track the object. There may be cases for input recognition orobject tracking wherein an ultrasound is provided without detectedsensation to the object.

Still referring to object device 100, storage device 110 may be any diskbased or solid state memory device for storing data. Power source 112may be a plug-in, battery, solar panels for receiving and storing solarenergy, or a device for receiving and storing wireless power asdescribed in U.S. Pat. No. 7,027,311 herein incorporated by reference asif fully set forth.

One or more network adapters 128 may be configured as a FrequencyDivision Multiple Access (FDMA), single carrier FDMA (SC-FDMA), TimeDivision Multiple Access (TDMA), Code Division Multiple Access (CDMA),Orthogonal Frequency-Division Multiplexing (OFDM), OrthogonalFrequency-Division Multiple Access (OFDMA), Global System for Mobile(GSM) communications, Interim Standard 95 (IS-95), IS-856, Enhanced Datarates for GSM Evolution (EDGE), General Packet Radio Service (GPRS),Universal Mobile Telecommunications System (UMTS), cdma2000, widebandCDMA (W-CDMA), High-Speed Downlink Packet Access (HSDPA), High-SpeedUplink Packet Access (HSUPA), High-Speed Packet Access (HSPA), EvolvedHSPA (HSPA+), Long Term Evolution (LTE), LTE Advanced (LTE-A), 802.11x,Wi-Fi, Zigbee, Ultra-WideBand (UWB), 802.16x, 802.15, Wi-Max, mobileWi-Max, Bluetooth, radio frequency identification (RFID), Infrared DataAssociation (IrDA), near-field communications (NFC), or any otherwireless or wired transceiver for modulating and demodulating signalsvia one or more antennas 130. One or more network adapters 128 may alsobe configured for automobile to automobile, car to car, vehicle tovehicle (V2V), or wireless access for vehicular environments (WAVE)communication. One or more network adapters 128 may also be configuredfor human body communications where the human body is used tocommunicate data between at least two computers coupled to the humanbody.

Any of devices, controllers, displays, components, etc. in object device100 may be combined, made integral, or separated as desired. Forinstance, elevation, indenting, or texturizing controller 121 may becombined with ultrasound source/detector 125 in one unit.

FIGS. 2a-2c are diagrams of configurations for providing a displaydevice with adaptive transparency. In FIG. 2a , display device 200comprises of an array of pixels 202 ₁-202 _(n), 204 ₁-204 _(n), and 206₁-206 _(n). Display device 200 can be either a flexible, bendable, orrigid display device for displaying video, images, photos, graphics,text, etc. Display device 200 may be configured for reflective,transmissive, transflective, passive matrix, or active matrix operation.Display device 200 may be configured with at least some of thecomponents and parts shown in object device 100.

FIG. 2b shows a different view of display device 200 having pixels layer208 and adaptive layer 210. Although two layers are shown in FIG. 2b ,display device 200 in FIG. 2b may be composed of a plurality of otherlayers having components or devices not shown. For instance, a mirrorlike layer may be included below adaptive layer 210. In addition, layers208 and 210 may be composed of sublayers having components or devicesnot shown. In FIG. 2b , the viewing may be from direction 211 and may beup to 180 degrees.

FIG. 2c is a diagram for providing a double-sided viewing display devicewith adaptive transparency. In FIG. 2c , adaptive layer 210 is situatedin between pixels layers 208 and 212. Although three layers are shown inFIG. 2c , display device 200 in FIG. 2c may be composed of a pluralityof other layers having components or devices not shown. In addition,layers 208, 210, and 212 may be composed of sublayers having componentsor devices not shown. In FIG. 2c , the viewing may be from directions214 or 216 and may be up to 180 degrees. Double-sided viewing may beused to display different information on each side of adaptive layer210. For instance, pixels layer 208 may show a video in a programmedzone or area while pixels layer 212 displays an image over the same ordifferent zone or area.

Adaptive layer 210 may comprise of electronically or electromechanicallycontrolled polarization filter devices. Adaptive layer 210 may also beany other devices for electronically or electromechanically controllingthe emission, transmission, reflection, refraction, or scattering ofelectromagnetic waves, such as light, through a layer thereby being ableto adaptively control transparency of a display. Examples ofelectronically or electromechanically controlled polarization aredescribed in U.S. Pat. Nos. 7,751,122 and 7,724,441 and U.S. PatentPublication Nos. 2010-171680 all herein incorporated by reference as iffully set forth. Alternatively, adaptive layer 210 may be comprised of asubstrate material that can be electronically controlled. An example ofsuch a material may be nanotechnology enabled nanoparticles.

Adaptive layer 210 may be divided into selectively controllablepredetermined/adaptive zones or areas of different sizes. A zone or areamay be comprised of tens, hundreds, thousands, millions, or any numberof scanned horizontal or vertical lines of pixels. The zones or areasmay be used for determining when a certain portion of display device 200is operating in a clear, semi-clear, see-through, partially see-through,transparent, semi-transparent, translucent, or semi-translucentoperating state or mode. These states or modes allow a user to see atleast partially the environment behind display device 200.

The zones or areas may be used for determining when a certain portion ofdisplay device 200 is operating in a nontranslucent, nontransparent,darkened, dark, opaque, or semi-opaque operating state or mode. Thesestates or modes inhibit or prevent a user to see at least partially theenvironment behind display device 200. Any of the states or modes listedabove may be combined in varying degrees in the given examples ordescriptions. Switching between different states or mode for an area orzone may be done based on configuration or control information providedto one or more displays controller 120 for properly displaying the areasor zones by adaptively controlling adaptive layer 210.

Having a transparent display device with adaptive transparency providesthe ability to selectively see any combination of reality, mixedreality, and normally displayed information with a darkened background.The switching between different operating states or modes may be neededdepending on the information being displayed. For augmented or mixedreality a clear, semi-clear, or equivalent state is useful foroverlaying, superimposing, covering up, or encompassing data over a realworld environment. However, for watching video a darkened or opaquebackground may be desirable to provide better contrast especially inenvironments with a high level of ambient light.

Moreover, the switching of different operating states or modes may bedone automatically based on feedback from sensors 126 or manually. Forinstance, sensors 126 may detect the amount of light in an environmentand adjust the transparency for a certain zone or area for optimalviewing. Under direct sunlight, display device 200 can be configured toadaptively block sun rays directed towards a user's eyes by making azone or area opaque or semi-opaque. The switching of different states ormodes may also be done automatically based on the information requestedto be displayed or context awareness feedback detected by object device100.

When display device 200 is configured with a mirror like layer orbackplane, the device may function as a mirror when adaptive layer 210is clear or semi-clear. Display device 200 may also be configured tofunction as an adaptive, interactive, smart, or augmented reality mirrorby displaying first or third person profile information on pixels layer208 when adaptive layer 210 is clear or semi-clear in any zone or area.Display device 200 may also be configured to operate as a mirror andsmart display device by displaying video, news, or any information inany zone or area of pixels layer 208 with adaptive layer 210 operatingin an opaque or semi-opaque mode.

FIG. 3 is a diagram of a displayed augmented reality environment.Display device 300 includes a clear display area 302. Although one sideis shown, display device 300 may be configured for double-sided viewingas shown in FIG. 2c . Display device 300 may be configured with at leastsome of the components and parts shown in object device 100.

Image or video 304 is displayed with a clear or semi-clear background.Substantially circular area or zone 306 may be controlled to provide anopaque or semi-opaque region. Although the example has an image orvideo, text or any other form of information may be displayed on displaydevice 300. Display device 300 may also have circuitry configured toprovide area or zone 316 where a border is opaque or semi-opaque whilespace or area 317 is clear or semi-clear. The configuration for area orzone 316 may be used for a picture frame, theater stage, classroom,interactive blackboard, presentations, trade show etc. where a user orobject may be in front or behind area 317 with information adaptivelybeing displayed in area or zone 316 and/or area 317.

In addition, substantially diamond like area or zone 318 may becontrolled to provide an opaque or semi-opaque region. Besides theshapes shown on display device 300, any controllably transparent shapemay be provided such as a square, quadrangle, polygon, triangle,quadrilateral, pentagon, hexagon, octagon, star, oval, sphere, etc.

Still referring to FIG. 3, image or video 308 is provided with an opaqueor semi-opaque substantially rectangular area or zone 310. Area or zone310 may be moved via path 312 to position 314 in response to a touchinput, multitouch inputs, any command, zoom command, eye trackingfeedback, blinking, facial gestures, hand gestures, voice command,speech command, brainwaves, motion, movement, location change, positionchange, or context awareness detected by pressure sensors 123, touchdetectors 124, sensors 126, and/or GNSS device 114. Area or zone 310 mayalso be moved via path 312 responsive to an anticipated gaze of a user.

Area or zone 310 may also be adaptively duplicated, removed, resized, orconverted to clear or semi-clear based on commands or inputs given inthe other examples. Moreover, area or zone 310 and/or image or video 308may be expanded, shrunk, zoomed in, or zoomed out via any user inputpreviously mentioned. As the size of area or zone 310 is adjusted,pixels are adaptively darkened or lightened in response to the sizechange of area or zone 310.

Elevation, indenting, or texturizing controller 121 may be used toprovide areas or zones with adaptive transparency at different levelsrelative to the top plane of display device 300 providing realtexturing. The different levels may be used to represent on-screenbuttons, special text, an advertisement, a game related object, etc.Similarly, ultrasound source/detector 125 may be used for providingaugmented or mixed reality in combination with the adaptive transparencyin display device 300 by projecting airborne ultrasound signals to anobject or part of a person, such as a finger or hand, related todisplayed information.

Alternatively, clear display area 302 may be inverted or reversed to bemostly semi-opaque or opaque having displayed information and a selectedwindow is clear. One benefit for this configuration is being able to seea person through a clear window in a meeting or office while being ableto read content on a screen or display at the same time.

Image or video 320 is overlaid, superimposed, or encompassed on anobject or person seen through clear display area 302 for creating andpresenting augmented or mixed reality based on other user or third partyinformation. The augmented object or person may also include anaugmented background object or scenery 321. Alternatively, rather thanbeing viewed through clear display area 302, image or video 320 may begenerated by a camera capturing the real world environment anddisplaying it with overlaid, superimposed, or encompassed information.The camera may be a separate unit or an integrated device into thedisplay area or pixels as described in U.S. Patent Publication Nos.2010-321355 and 2011-018850 both herein incorporated by reference as iffully set forth. In addition to image or video, the overlaid informationmay be text, email, instant messages, audio, music, sounds, etc.

The other user or third party information may be received wirelesslyusing one or more network adapters 128 in a message from a server,directly from another user device, or a “cloud” based system (notshown). The other user or third party information may also be receivedby beacons transmitting data close to display device 300. The other useror third party information may be stored and accessed on storage device110.

The other user information may be based on a profile, a user profile,social networking profiles, dating profiles, any user setting, a gamesetting, a mood profile, a magazine style, a custom look, mimicking acharacter, emulating a character, a Second Life setting, virtualFacebook, augmented Facebook, etc. of another user to provide the viewof the world onto a first or primary user of object device 100. Bytailoring another's environment and controlling their environment newinsights are gained and learned from others. The other user may be afriend, relative, stranger, coworker, etc. of the first or primary user.

Moreover, profile settings may exist to adapt to different moods,relationship statuses, events, environments, etc. set by another user orentity. For instance, the another user may be seen by others as a dragonif they are angry, a geek to look smart, or a buff guy to arouse womenif they are single. Moreover, the profile may be configured to changethe augmentation of a person, environment, or object based on time, daysof the week, seasons, holidays, etc.

With context awareness, a profile can adapt based on a detected event orenvironment. Another user's profile or information may say every treethat is detected by the object device 100 and/or display device 300 isautomatically augmented to appear as a palm tree. This action is similarto an automatic find and replace feature. The other profile orinformation may also control how you see other people. For instance,certain people another person does not like turns into a monkey ordevil. The profile may also have settings to augment a background orscenery in addition to an object or person itself to be displayed ondisplay device 300, such as with background object or scenery 321.

The other user information or profile may be received and generated inthe augmented or mixed reality world when the other user or an object isdetected within the environment of the first or primary user of objectdevice 100 using well-known techniques for text recognition, characterrecognition, image recognition, video recognition, speech recognition,or voice recognition processed by one or more processors 102 or remotelyon a server (not shown). The other user may also be detected in theaugment reality world using location based services (LBS) with GNSSdevice 114. Moreover, the other user may be detected based on a smart orRFID tag detected by sensors 126.

As another example, the other user information may set that today theuser will be seen by others as a certain movie star with selectablemakeup, clothes, or hairstyle. Another user may set a likeness, face, orcharacter to be played. As another example, another user may set how acertain food or car will look to others.

As another example, other user information may be purchased in a virtualmarketplace or obtained for free by selling advertisements on theaugmented person or object. Alternatively for e-business, another usermay create a custom augmented reality design and offer it to sell to thefirst or primary user. Moreover, a user may be paid by a sponsor to looka certain way to other users or devices with a sponsorship profile. Amunicipality or owner of real estate may sell space or objects seen inpublic to a sponsor to be augmented to selectively be seen a certain wayby other users or devices.

Although the examples given thus far are from a second person point ofview, the present embodiments may also create first, third, oralternating party augmented reality worlds. In particular with thirdparty information, a user may be able to see an augmented realityversion of a show or program. Third party information may also include agaming augmented reality environment provided by a service provider.Moreover, any of the example information provided above for other usersmay be provided by a third party or service provider. Thus, the presentexamples can be used with any narrative mode.

In the case of first party augmented reality profile, the profile mayset as an example how a person's house or car looks to them. Forinstance, a house may be augmented to appear as a castle and an old carmay be augmented to appear as a sports car. This may be extended to anyother possession or object. The first person profile may then be sharedso that others can see their possessions in the same way when present inthe same environment and detected by object device 100.

For the case of augmented audio, music, or sounds, augmented reality canbe used to play a song associated with another user or object when thatuser or object is detected by object device 100. The augmented audio,music, or sounds may be included in the other user information or anyprofile. For instance, image or video 320 may include a selected themesong to be played when the user or object is in the environment of thefirst or primary user. This example is similar to an augmented realityringtone or videotone.

As another example, display device 300 can be configured to display anaugmented theme, environment, drink, or food in a restaurant profile.For instance, a restaurant may be designed with plain walls or simpledecorations to be augmented. Augmented reality devices similar to objectdevice 100 may be handed to patrons when entering a restaurant.Alternatively, an augmented reality capable device downloads therestaurant profile from a server on the Internet or a beacon in therestaurant using one or more network adapters 128. A benefit to this maybe that a restaurant has lower startup costs and can have adaptivedecorations to generate new business. A similar configuration may beused for a decorated restaurant where the augmented reality enhances thedining experience.

As another example, the augmented reality other user information orprofile may include motion, movement, or action based setting. Forinstance, another user may be able to be seen as lifting objects up whenmaking the motion, movement, or action of bending down to lift a car.The motion, movement, or action can be anything detected by objectdevice 100 and/or sensors 126.

Furthermore, for inputting data to object device 100 for adaptivetransparency, augmented reality, or any other application, camera in I/Odevices 118 may be used to detect or read lip, mouth, or tongue movementfor speech recognition, voice recognition, or voice commands. Lip,mouth, or tongue movement may be detected when the user is speaking withsound or silently speaking without sound. Speaking without sound,speaking with little sound, pretend speaking, or whispering isadvantageous since it does not disturb others around the user. Imagescaptured by camera in I/O devices 118 may be processed by well-knowntechniques by image or video recognition engines in software 108 todetermine user input. Object device 100 may use lip or tongue movementprimarily for inputting text, to assist with an existing speech or voicerecognition system to interpret spoken language, or to determine voicecommands.

In addition, data may be inputted or action taken depending on howobject device 100 is held based on finger placement to provide a contextaware operating system. For instance when object device 100 isconfigured as augmented reality glasses, holding the glasses like acamera will put object device 100 into camera mode to take a picture.Object device 100 may detect finger placement by having a touchsensitive case or shell. As another example, when object device 100determines it is outside and moving, the operating system willautomatically switch to navigation mode.

As mentioned above, display device 300 may be part of augmented realityglasses, goggles, or a head mounted display having at least some of thecomponents in object device 100. Glasses or goggles are advantageoussince it allows hands free operation. In addition, picoprojector 132 maybe used to project information based on other user information on to anysurface. Sensors 126 may be configured to update the augmented realityenvironment or adaptive transparency of display device 300 based onmotion or rotation. The present examples are also applicable for virtualreality worlds. Moreover, a haptic or sensory feedback glove or braceletmay be provided to augment the feeling of an environment.

FIG. 4 is a process 400 for providing a display device with adaptivetransparency. Configuration or control information (402) is receivedsetting what areas or zones on a display device will selectively betransparent, opaque, or any level in between. The configuration orcontrol information may be received remotely or locally on object device100. As an example, the configuration or control information may bereceived from a server remotely via one or more network adapters 128 orfrom software residing on object device 100. The configuration orcontrol information may be used by one or more displays controller 120for properly displaying the areas or zones by adaptively controllingadaptive layer 210. The pixels and background for the area or zone areconfigured depending on the configuration or control information (404).The pixels and background may then be updated with new configuration orcontrol information (406) to combine, move, resize, refresh, or alterareas or zones.

FIG. 5 is a process 500 for providing an augmented reality environment.The other user or object is detected in an environment (502), forexample by image recognition. Other user or third party information,such as a profile, is received (504). The other user or third partyinformation is used to update the displayed information on a displaydevice (506), that may be adaptively transparent, to augment an objector person. The displayed augmented reality world may then be updatedbased on motion, such as provided by the other user or third partyinformation (508). Optionally, feedback is provided for the augmentedlook to the other user and/or the other user is engaged (510).

Although features and elements are described above in particularcombinations, each feature or element may be used alone without theother features and elements or in various combinations with or withoutother features and elements. The methods, processes, or flow chartsprovided herein may be implemented in a computer program, software, orfirmware incorporated in a computer-readable storage medium forexecution by a general purpose computer or a processor. Examples ofcomputer-readable storage mediums include a read only memory (ROM), arandom access memory (RAM), a register, cache memory, semiconductormemory devices, magnetic media such as internal hard disks and removabledisks, magneto-optical media, and optical media such as CD-ROM disks,digital versatile disks (DVDs), and BluRay discs.

Suitable processors include, by way of example, a general purposeprocessor, a multicore processor, a special purpose processor, aconventional processor, a digital signal processor (DSP), a plurality ofmicroprocessors, one or more microprocessors in association with a DSPcore, a controller, a microcontroller, Application Specific IntegratedCircuits (ASICs), Field Programmable Gate Arrays (FPGAs) circuits, anyother type of integrated circuit (IC), and/or a state machine.

A processor in association with software may be used to implementhardware functions for use in a computer, wireless transmit receive unit(WTRU), or any host computer. The programmed hardware functions may beused in conjunction with modules, implemented in hardware and/orsoftware, such as a camera, a video camera module, a videophone, aspeakerphone, a vibration device, a speaker, a microphone, a televisiontransceiver, a hands free headset, a keyboard, a Bluetooth® module, afrequency modulated (FM) radio unit, a liquid crystal display (LCD)display unit, an organic light-emitting diode (OLED) display unit, adigital music player, a media player, a video game player module, anInternet browser, and/or any wireless local area network (WLAN) or UltraWide Band (UWB) module.

Any of the displays, processors, memories, devices or any othercomponent disclosed may be configured, produced, or engineered usingnanotechology based nanoparticles or nanodevices.

What is claimed is:
 1. A mobile electronic device comprising: atransceiver configured to receive, via a network, a profile and custommusic set by a particular user, wherein the received profile is from asocial network application and used to determine a current view with thecustom music provided by the mobile electronic device to a user of themobile electronic device; a camera is configured to capture video of thecurrent view, wherein the particular user in the video of the currentview is detected and augmented substantially in real time by a processorby overlaying images associated with a virtual character specified inthe received profile to mimic the virtual character in an augmentedvideo mixed with the custom music; a multi-touch display device isconfigured to display the augmented video and an audio device configuredto play the mixed in custom music set by a particular user to the user;and wherein the multi-touch display device comprises a transparentdisplay area and wherein a depth level and shape of the transparentdisplay area relative to a substantially top portion of the multi-touchdisplay device is adaptively made see-through in the augmented video toproduce a reality view by a multi-touch input.
 2. A method comprising:receiving, by a mobile electronic device via a network, a profile andcustom music set by a particular user, wherein the received profile isfrom a social network application and used to determine a current viewwith the custom music provided by the mobile electronic device to a userof the mobile electronic device; capturing, by a camera of the mobileelectronic device, video of the current view, wherein the particularuser in the video of the current view is detected and augmentedsubstantially in real time by a processor by overlaying imagesassociated with a virtual character specified in the received profile tomimic the virtual character in an augmented video mixed with the custommusic; displaying, by a multi-touch display device, the augmented videoand playing the mixed in custom music set by a particular user to theuser; and wherein the multi-touch display device comprises a transparentdisplay area and wherein a depth level and shape of the transparentdisplay area relative to a substantially top portion of the multi-touchdisplay device is adaptively made see-through in the augmented video toproduce a reality view by a multi-touch input.